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Abstract:  Reliability  studies  in  terms  of  probability  of  detection  (POD)  evaluation  are  a  critical  component  of 
nondestructive  evaluation  (NDE)  inspection  system  validation.  POD  studies  provide  engineers  with  data  on  the 
reliability  of  detecting  cracks  across  a  distribution  of  sizes  that  can  be  used  as  an  input  to  probabilistic  risk 
assessment  analysis.  Traditional  POD  evaluation  methodologies  are  entirely  empirical.  In  most  cases,  the  cost 
of  manufacturing  the  number  of  samples  required  for  a  traditional  POD  study  is  prohibitive  and  may  delay  or 
prevent  a  new  inspection  procedure  or  new  technology  from  being  implemented.  A  new  strategy  for  the  design 
and  execution  of  POD  studies  has  been  discussed  using  a  model-assisted  POD  (MAPOD)  approach.  A 
demonstration  study  is  presented  for  a  class  of  aerospace  structural  inspection  problems  and  considers  the  use  of 
computational  electromagnetic  models  and  new  statistical  analysis  methods  in  the  POD  evaluation.  The  volume 
integral  method  was  successfully  used  to  simulate  eddy  current  measurements  for  varying  crack  length  around 
fastener  holes  in  a  two-layer  aluminum  structure.  Good  agreement  was  achieved  between  experimental  and  full- 
model  assisted  (FMA)  approaches. 

Keywords:  cracks,  eddy  current,  probability  of  detection,  reliability,  volume  integral  method. 


1.  Introduction 

New  nondestructive  inspection  (NDI)  techniques  continue  to  be  developed  and  applied  to  address  new 
inspection  requirements  and/or  improve  existing  techniques.  To  validate  the  performance  of  new  inspection 
techniques,  Probability  of  Detection  (POD)  studies  can  be  performed.  A  POD  study  includes  preparation  of 
multiple  samples  that  closely  mimic  the  geometry  of  the  structure  to  be  inspected.  These  samples  include  a 
statistically  significant  number  of  inspection  opportunities  containing  a  distribution  of  flaw  sizes  and  locations 
anticipated  in  the  structure.  The  preparation  of  POD  samples  and  the  process  to  acquire  data  from  these  samples 
is  often  very  time  consuming  and  expensive.  This  is  especially  true  for  samples  that  include  fatigue  cracks, 
which  should  be  grown  using  load  profiles  typically  seen  by  the  structure  of  interest  to  accurately  represent 
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fatigue  cracks  that  could  be  present.  The  financial  and  time  burden  can  become  significant  hurdles  in  the 
development  and  execution  of  POD  studies  when  validating  new  inspection  techniques. 

A  new  strategy  for  the  design  and  execution  of  POD  studies  has  been  proposed  [1]  and  recently  advanced  as 
a  part  of  the  Model-Assisted  POD  (MAPOD)  Working  Group  [2].  One  MAPOD  approach  includes  the  use  of 
transfer  functions  to  enable  Electric  Discharge  Machined  (EDM)  notches  to  be  used  as  substitutes  for  grown 
fatigue  cracks  while  minimizing  uncertainty  in  the  sensitivity  of  the  detection  process.  However,  this  requires 
preparation  of  a  large  number  of  test  samples.  An  alternative  MAPOD  approach  uses  advanced  computer 
simulations  to  model  the  inspection  process  and  determines  the  POD  for  the  inspection  technique  through  a 
combination  of  experimental  and  simulated  data.  It  is  important  to  note  that  these  models  must  address 
parameters  in  the  structure  that  can  affect  the  interaction  of  the  inspection  method  with  the  structure,  such  as 
tapers,  multiple  layers,  air  gaps,  fastener  types,  fastener  fit,  and  other  variables  introduced  by  design, 
manufacture,  and/or  previous  repairs.  Using  computer  and  empirical  models  to  address  these  variables  that 
cannot  be  easily  recreated  in  experimental  samples  is  a  significant  opportunity. 

2.  Reliability  Demonstration  Using  Model-Assisted  POD  Evaluation 

A  demonstration  is  presented  for  a  full  model-assisted  (FMA)  POD  methodology  incorporating  computer 
simulation  for  the  inspection  of  cracks  around  fastener  sites  in  a  two  layer  aircraft  wing-type  structure  inspection 
performed  with  an  eddy  current  technique  [3].  The  demonstration  follows  a  proposed  FMA  protocol  [2] 
including  the  following  steps:  1)  identify  the  scope  of  the  POD  study,  2)  identify  factors  that  control  signal  and 
noise,  3)  evaluate  quality  of  physics-based  models,  4)  acquire,  develop,  and  validate  simulation  tools,  5)  acquire 
input  parameters  and  parameter  distributions,  6)  conduct  flaw  signal  distribution  simulations  and  noise  signal 
distribution  simulations,  7)  acquire  remaining  information  on  factors  empirically,  8)  acquire  marginal 
information  on  independent  factors,  9)  acquire  covariance  information  on  dependent  factors,  10)  combine  6,  8, 
and  9  into  full  signal  and  noise  distributions,  and  lastly  11)  compute  POD  curve  and  probability  of  false  call  rate 
(POFC). 

To  validate  the  proposed  model-assisted  POD  methodology,  a  set  of  experimental  data  for  hidden  cracks 
around  fasteners  in  multilayer  structures  was  analyzed  [4].  The  first  and  second  layer  thickness  values  were 
0.156"  and  0.100"  respectively  and  made  of  7075-T651  aluminum.  Real  fatigue  cracks  were  manufactured 
varying  from  0.020"-0. 169".  Raster  scan  data  was  acquired  from  these  samples  using  a  conventional  eddy 
current  probe  at  600  Hz.  New  feature  extraction  and  automated  signal  classification  algorithms  were  developed 
and  applied  to  the  experimental  data  set  demonstrating  the  ability  to  detect  small  cracks  around  fasteners  while 
maintaining  a  low  false  call  rate  [4]. 

Simulated  studies  were  also  performed  incoiporating  the  experimental  problem  geometry  of  an  eddy  current 
inspection  of  a  two  layer  structure  with  a  fastener  site  and  a  crack  [4].  Parametric  studies  were  performed 
investigating  varying  crack  size  and  type,  considering  both  through  and  comer  crack  geometries  (Figure  1). 
VIC-3D®,  a  commercial  software  package  based  on  the  volume  integral  method  [5],  was  used  to  perfomi  the 
simulations  and  found  to  address  a  great  majority  of  the  model  variables.  The  simulated  and  experimental  data 
for  a  given  probe  and  inspection  geometry  were  equated  using  a  calibration  procedure  developed  in  prior  work 
[6].  Figure  2  presents  the  calibration  results  demonstrating  good  agreement  between  the  model  and  simulated 
data  for  the  no-crack  fastener  site.  Figure  3  presents  a  comparison  of  the  simulated  and  model  data  for  varying 
crack  length  and  crack  type  (comer,  through).  In  general,  there  is  good  agreement  between  the  two  data  sets 
with  some  error  for  select  experimental  data  points.  Figure  4(a)  presents  the  experimental  data  including  the 
response  for  unflawed  specimens.  The  distribution  of  the  unflawed  data  shown  in  Figure  4(b)  was  used  to 
represent  the  noise  distribution  of  the  experimental  eddy  current  measurement  system. 

A  Monte  Carlo  simulation  was  performed  using  the  noise  distribution  for  the  unflawed  fastener  sites,  the 
comer  crack  model,  and  the  through  crack  model,  to  populate  the  full-model  assisted  data  sets  for  POD 
evaluation.  Additional  details  on  the  FMA  methodology  applied  to  this  problem  can  be  found  in  [3].  Figure  5 
presents  a  comparison  of  the  POD  results  for  the  experimental  and  full-model  assisted  approaches.  A  good 
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match  was  achieved  between  experimental  and  full-model  assisted  (FMA)  approaches  where  the  FMA  POD 
curve  was  found  to  be  within  the  confidence  bounds  (95%)  of  the  empirical  POD  curve.  This  good  result  was 
achieved  while  limiting  the  experimental  measurement  data  included  in  the  noise  distribution  evaluation  for  the 
model-assisted  evaluation  from  only  unflawed  specimens. 
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Figure  1 .  Diagram  of  fastener  site  model  with  (a)  a  comer  crack  and  (b)  a  through  crack. 


Figure  2.  Calibration  fit  of  model  with  respect  to  experimental  data  for  titanium  fastener  site  (X,  Y  in  Volts). 
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Figure  3.  Comparison  of  experimental  and  simulated  data  for  varying  crack  length  and  type. 


Figure  4.  (a)  Experimental  data  including  unflawed  responses  with  the  corresponding  (b)  probability  density 

function  for  unflawed  data. 
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Figure  5.  POD  evaluation  results  for  experimental  and  full  model-assisted  POD  studies. 


3.  Results  and  Conclusions 

The  feasibility  of  using  a  model-assisted  POD  evaluation  approach  was  demonstrated  for  eddy  current  NDE 
techniques  applied  to  aerospace  structures.  A  good  match  was  achieved  between  experimental  and  full-model 
assisted  (FMA)  approaches  where  the  FMA  POD  curve  was  found  to  be  within  the  confidence  bounds  (95%)  of 
the  empirical  POD  curve.  An  advantage  of  the  FMA  approach  was  demonstrated  by  only  requiring  empirical 
data  from  unflawed  samples,  indicating  the  potential  for  cost  savings  associated  with  minimizing  the 
manufacture  of  samples  with  real  flaws.  This  model-assisted  approach  can  also  likely  be  extended  to  address 
problems  of  varying  fastener  diameter,  thickness  of  multiple  layers,  crack  location,  scan  plan  and  probe  designs. 
Flowever,  significant  challenges  remain  in  developing  MAPOD,  including  improvements  to  the  existing 
computer  modeling  software,  incorporation  of  multiple  structural  variables  to  accurately  determine  the  damage 
in  the  structure,  optimization  of  the  methodology  to  combine  both  test  samples  and  computer  simulations,  and 
determination  of  the  uncertainty  in  the  inspection  process  when  both  experimental  and  computer  simulation  data 
is  used. 
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